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ABSTRACT 


This  report  examines  some  statistical  properties  of  a  first-order, 
stationary  time  series  (xt }  and  of  a  series  fy consisting  of  disjoint 
sums  of  (xtJ.  A  connection  between  continuous  stochastic  processes  and 
discrete  time  series  is  made  via  the  concept  of  correlation  time  interval. 
Expressions  for  the  variances  of  the  sum  and  average  of  {x^.}  for  n  terms 
are  derived.  An  asymptotic  variance  estimate  of  the  average  is  derived 
and  is  shown  to  be  a  reasonable  upper  bound. 

General  expressions  are  derived  for  the  variance  and  autocovariance 
of  the  integrated  process  {y^.}  in  terms  of  the  same  properties  for  any 
stationary  process  {x^.}.  These  results  are  particularized  for  the  case 
in  which  { x^. >  is  first-order.  Expressions  for  the  spectral  densities 
of  Xj.  and  y^  are  given  when  (x^l  is  first-order.  A  computer  program  to 
calculate  the  autocovariance  and  autospectrum  of  y^  is  attached. 


EXECUTIVE  SUMMARY 


This  report  is  addressed  to  analysts  with  a  good  background  in  statistics 
and  with  some  exposure  to  time  series  analysis.  Applications  of  time  series 
are  found  in  a  host  of  fields  including  the  physical  and  biological  sciences, 
in  economics,  and  in  industrial  processes.  The  principal  products  of  this 
study  are  derivations  of  certain  equations  and  numerical  illustrations  of 
their  use.  A  computer  program  is  provided  to  implement  the  calculations. 
These  results  relate  statistical  properties  of  one  time  series  to  another 
series  whose  elements  are  subsummed  from  those  of  the  first.  Special 
attention  is  given  the  case  in  which  the  first  series  is  related  to  itself 
with  first-order  dynamics.  The  fluctuating  behavior  of  the  series  studied 
is  assumed  to  be  steady-state.  The  study  results  can  assist  in  building 
statistical  models  thru  identification  of  proper  functional  forms  and  in 
parameter  estimation. 
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MEMORANDUM  REPORT 

SUBJECT:  Statistical  Properties  of  an  Integrated,  Stationary  Stochastic  Process 

1.  Background 

In  some  applications  where  a  random  time  series  is  studied,  the  questions 
of  interest  are:  What  is  the  variance  of  the  series  sum?  and  What  is  the 
variance  of  the  average?  In  this  kind  of  application  a  member  of  the  time 
series  is  viewed  as  the  rate  of  change  of  the  quantity  of  interest.  Such 
might  be  the  case,  for  example,  in  a  series  of  production  output  rates. 

The  focus  in  this  case  is  on  the  output  of  the  process  over  some  long  time 
interval.  The  latter  is,  of  course,  the  integral  (or  numerical  sum)  of  the 
rate  over  this  time  period.  Perhaps  one  cannot  be  assured  that  the  rates 
are  statistically  independent.  Or  perhaps  --  as  is  often  the  case  in  a  real 
manufacturing  process  --  the  rates  are  known  to  be  autocorrelated.  In  this 
case  one  might  ask  how  precise  is  the  estimate  of  the  quantity  of  interest. 

An  example  of  an  autocorrelated  industrial  process  of  military  interest  is 
the  production  of  batches  of  explosive  Composition  B  [1],  Because  of  RDX- 
batch  mixing  operations,  successive  batches  of  Comp  B  exhibit  batch  viscosities 
which  are  not  statistically  independent. 

Other  applications  may  require  the  identification  of  an  appropriate 
stochastic  model  for  a  time  series  (yt }  subsummed  from  another  {x.}. 

Successive  members  of  {yt>  do  not  share  members  of  {x. } .  For  example,  weekly 
totals  are  obtained  from  daily  quantities.  Questions  of  interest  in  this 
regard  might  be:  How  does  the  autocorrelation  function  of  y  compare  with 
that  of  x.?  or  How  do  the  spectra  of  these  related  processes  compare? 


[1]  "Stochastic  Model  of  the  Batch  Viscosity  Generation  Process  in 
Sequential  Batches  of  Composition  B  ...",  DRSAR/PE/N-88, 

(AD  B057395) ,  April  1981. 
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Paraphrasing,  one  might  ask  how  independent  are  (y^l  relative  to  {x^l  or, 
relatively,  how  variable  are  the  fluctuations  of  a  particular  duration  in 
the  two  series.  Another  challenging  problem  arises  when  only  data  for 
subsummed  series  are  available  but  one  wishes  to  model  { x^. } .  An  example 
of  a  stationary  autocorrel ated  time  series,  in  which  subsummed  data  is 
often  published,  is  found  in  ballistic  tests  of  large  caliber  artillery, 

[2].  In  some  instances  evidence  of  "ballistic  memory"  exists. 

2.  Objectives 

One  purpose  of  this  report  is  to  derive  an  expression  for  the  variance  of 
the  integral  of  a  stationary  stochastic  process  which  is  taken  to  be  first- 
order.  This  brief  exposition  will  serve  to  quantify  the  effect  of  auto¬ 
correlation  in  the  rate  process  on  the  precision  of  sums  (or  integrals)  and 
averages  of  that  process.  Quantitative  insights  will  be  provided  in  the 
discussion.  Another  objective  is  to  derive  expressions  for  the  autocorrelation 
function  and  the  autospectrum  of  a  first-order  process  { x^. }  and  of  the  subsummed 
process  fy^J -  These  are  compared  in  specific  numerical  examples. 

3.  Notation 

The  notational  conventions  of  Jenkins  and  Watts  [3]  are  followed  here. 
Consider  the  time  series  {x^.}  with  integer  index  t.  The  time  interval 
separating  successive  members  of  the  series  is  a  constant  A.  If  this 


[2]  "Statistical  Evidence  of  a  Memory  Process  in  Cannon  Artillery  Weapons 
and  Its  Implication  for  Testing,"  DRSAR/SA/N-76,  (AD  C013498) , 

January  1978. 

[3]  Jenkins,  G.M.  and  Watts,  D.G.  Spectral  Analysis  and  Its  Applications, 
Holden-Day,  San  Francisco,  c.  1969. 
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series  is  derived  from  a  continuous  process,  a  is  the  sampling  interval. 

This  process  is  assumed  covariance  stationary.  Without  loss  of  generality 
the  mean  of  the  process  is  taken  as  zero.  Thus,  with  E  denoting  the 
ensemble  expectation  operator, 

E[xt]  =  0  .  (!) 

The  variance,  V [x, ] ,  is  denoted  by  y  (0). 

v  XX 

Yxx(0)  =  E[xtI  '  (2) 

The  process  studied  satisfies  the  first-order  difference  equation: 

xt  =  bxt-l  +  nt  »  ,  (3a) 

with  constant  b  and  where 
{ n^.}  is  a  white  noise. 


Thus, 

E[nt ]  =  0 

Ynn(k>  =  EE"t  nt+k]  =  0  •  M  ' 

Ynn(0)  *  “Z  • 

Notationally,  let 

Yxn(k>  ’  ' 

Since  future  values  of  the  noise  in  (3a)  are  uncorrelated  with  xt, 
Yxn(k)  =  0  »  k<0  . 

The  autocorrelation  of  {x.}  is  denoted  by  p  (k),  where 

t  XX 

pxx(k)  =  Yxx(k)/Yxx(0)  • 


(3b) 

(3c) 

(3d) 

(4) 

(5) 

(6) 
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4.  Autocorrelation  of  x. 


From  (3a), 

E[xt+k  xtJ  =  EKbx^.j  +  nt+k)(bxt_j  +  nt)J  , 

Using  the  results  of  equations  (3)  with  the  above, 

v„„0<)  -  — 2  T.„(k-1) 


k>l 


XX' 


1-b 


xn1 


Clearly,  for  stability  of  the  {x^.}  process  |b|  <  1  . 

By  multiplying  both  sides  of  (3a)  by  and  taking  expectations, 

Y  (k)  =  b  y  (k-1)  +  y  (k) 

Txnx  1  rxnv  1  rnnv  1 

f 

Using  (3c)  and  (8)  , 

y  (k)  =  y  (0)  . 

'xn'  '  'xn 

With  (5),  equation  (8)  implies  that 

y  (0)  =y  (0)  =o^  . 

'xn'  ‘  'nn'  ' 

Hence,  from  (9) 

Yxn(k)=bka2  . 

Finally,  (7)  and  (11)  yield 

m  bk  2 
Yxx  ^ 


from  which 


1-b 

k 


Pxx<k>=b  , 

2 

with  the  variance  (ox  )  of  {x^}  given  by 


Yxx(0) 


1-b 


1  2 

T  0 


(7) 

(8) 

(9) 

(10) 
(11) 

(12) 

(13a) 

(13b) 
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5.  Correlation  Time  Interval 

A  concept  of  considerable  significance  in  time  series  analysis  is  correlation 
time,  t*.  This  notion  is  discussed  at  length  by  Fishman  and  Kiviat*  in  applica¬ 
tion  to  analysis  of  stochastic  simulations.  Any  stationary  stochastic  process 
must  be  observed  for  a  time  interval  of  2t*  in  order  to  derive  information  that 
is  equivalent  to  one  observation  of  an  uncorrelated  random  variable.  Thus,  the 
number  of  equivalent  degrees  of  freedom  obtained  by  observing  a  stationary 
process  for  time  T  is  T/(2t*).  For  a  continuous  lst-order  process,  i.e.,  one 
that  is  defined  at  every  point  in  time  --  say,  z(t)  --  the  autocorrelation 
function  is  shown  by  Fishman  and  Kiviat  to  have  the  form 

pzz(t)  =  e~U  ’  (14) 

where  A  is  a  rate  constant.  Now,  by  definition,  for  a  continuous  process 

Tz  ■  f0  dt  ■  (15) 

Then,  for  the  first-order  continuous  stochastic  process 

t*  =  1/A  .  (16) 

Similarly,  for  a  discrete  process,  the  correlation  time  is  defined  as 

Tx  =  AEk=0  pxx^  5  ( 1?) 

where  neighboring  members  of  {x^}  are  separated  by  duration  A.  Using 
(13a)  and  (17),  the  discrete  first-order  process  has  correlation  time 

t*  =  a/ ( 1—  b)  .  (18) 

It  is  worthwhile  to  compare  the  results  of  (16)  and  (18). 

Suppose  that  the  continuous  process  {z(t ) }  is  time-sampled  with  sampling 
interval  A.  In  this  case  the  process  is  observed  only  at  times  t  =  kA, 
with  k  an  integer. 


*  Fishman,  G.S.  and  Kiviat,  P.J.  Spectral  Analysis  of  Time  Series  Generated 
by  Simulation  Models,  RM-4393-PR,  Rand  Corp.,  Santa  Monica,  CA, 

February  1965. 
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(19a) 


Then,  the  correlation  function  in  (14)  would  be  defined  discretely  as 

Pzz(k)  =  bk 

with 

b  =  e'AA  .  (19b) 

The  constant  b  in  the  discrete  process  is  related  to  the  rate  parameter 
A  of  the  continuous  process  by 

A  =  -In  b/A  .  (20) 

Using  (16)  and  (20)  , 

t*/ A  =  -1/1 n  b  .  (21) 

The  last  equation  should  be  compared  with  its  counterpart  for  the  discrete 
process,  equation  (18).  To  give  a  numerical  example,  suppose  that  b  =  0.9. 
Then,  the  correlation  time  of  the  discrete  process,  from  (18),  is  10a.  For 
a  continuous  process  having  the  same  autocorrelation,  the  correlation  time, 
from  (21),  is  9.491a.  One  observes  that  the  value  of  x*  is  slightly  larger 
for  the  discrete  process.  This  would  require  a  longer  observation  period 
for  the  discrete  (time-sampled)  process  than  for  the  underlying  continuous 
stochastic  process  to  achieve  the  same  degree  of  statistical  precision. 

This  result  quantifies  the  information  loss  associated  with  coarse  time¬ 
sampling  of  a  continuous  process. 

6.  Variance  of  the  Sum  of  {x^.} 

After  the  slight  digression  in  paragraph  5,  we  return  to  the  main  task  of 
deriving  an  expression  for  the  variance  of  the  sum  of  the  discrete  time 
series  } - 
Let 

yt  =  Ek=l  xk  *  ^ 

Since  {x. >  has  zero  mean, 

E[yt]  =0  (23a) 

and 

V[yt]  =  E [yf]  .  (23b) 
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(24) 
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The  variance  of  {y , }  is  also  denoted  by  a  .  With  (22)  and  (23b) } 
t  y 

Also 

v[yt+1i  =  E[£l=i  +  Eixt+i) 

+  £1  xixi+k] +  2Ei=i  EIxixt+i|  •  (25> 

By  comparing  (24)  and  (25) 

V‘W  =  V[n!  +  xxx<°>  +  2Ek=lTxx<k>  •  ‘i1  •  <26> 

With  (12),  (13),  and  (26), 

V[yt+1]  =  V [y t ]  +  o*  (l+2et)  ,  t>l  ,  (27a) 

where 

Bt  =  4=1  bk  (27b) 


or 


Bt  =  b(l-bt)/(l-b)  . 

(27c) 

From  (22), 

viyp  - 

(28) 

From  (27a)  and  (28),  by  induction, 

V[yt]/o£  =  t  +  2zJ;}  6k  ,  t>2  . 

(29) 

Using  (27c)  the  last  term  on  the  right-hand  side  of  (29) 

can  be  written  as 

C  -  2b  [t  (l-b‘> , 

5t  1-b  1-b  J  • 

(30) 

Finally, 

4 /»x  =  *  +  st  • 

(31) 
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This  is  one  of  the  expressions  of  interest.  Note  that  an  uncorrelated  series, 
with  b=0,  would  produce  the  familiar  result  that  the  variance  of  the  sum  is 
the  sum  of  the  variances,  i.e.. 


The  effect  of  autocorrelation  on  the  ratio  oy/ax  is  to  increase  the 
effective  series  length  relative  to  an  uncorrelated  series  by  S^. 

As  noted  in  pgf .  4,  the  magnitude  of  b  must  be  less  than  unity. 

7.  Variance  of  the  Average  of  {x^} 

Suppose  that  the  variable  of  interest  is  the  estimated  mean  ( x of  the 


time  series  {x^},  where 

*t  '  hH  '  (33) 

Since  t  is  a  constant  in  (33), 

V[xt]  =  V[yt]/t2  .  (34) 

From  (31)  and  (34) , 

V[xt]  =  a2  (t  +  St)/t2  .  (35) 

If  the  {xt>  process  is  uncorrelated,  one  has  the  conventional  result 

V[xt]  =  cr2/t  .  (36) 


However,  for  long  correlation  intervals  and  relatively  short  sampling 
intervals,  one  may  expect  St  to  be  larger  than  t.  In  this  case  the 
variance  of  the  average  should  be  calculated  from  (30)  and  (35)  using 
the  best  estimate  of  b. 

3.  An  Asymptotic  Result 

As  t  becomes  large,  the  second  term  on  the  right  in  (30)  becomes  small 
relative  to  the  first.  Asymptotically,  the  variance  of  the  estimated  mean, 
given  in  (35),  becomes 

V[xt]  =  (a2/t)(l+b)/(l-b)  .  (37) 
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Thus,  for  a  positively  autocorrelated  first-order  time  series,  the  variance 
of  the  average  is  amplified  by  a  factor  of  at  most  (l+b)/(l-b)  relative  to 
that  of  an  uncorrelated  time  series  of  the  same  length. 

9.  A  Numerical  Example 

To  provide  a  feeling  for  magnitude  relative  to  this  problem,  consider 
the  following  special  case.  Suppose  that  {xt>  is  a  first-order  time  series 
obtained  by  sampling  a  continuous  stochastic  process  every  minute.  The 
continuous  process  is  claimed  to  have  a  correlation  time  of  20  minutes. 

This  implies  that  every  40  samples  are  "worth"  one  degree  of  freedom. 

From  (18),  solving  for  b  gives 
b  =  0.95  . 

Then,  the  variance  of  the  estimated  mean  is  given  by  (30)  and  (35)  as 
o2x  [t  +  38  (t  -  20  (1  -  0.95t))]/t2  . 

The  variance  of  an  uncorrelated  series  of  length  t  is,  by  contrast, 

a2/ 1  . 

X 

The  asymptotic  variance  estimate  for  this  example,  from  (37),  is 

394n  ■ 

To  yield  a  reasonable  approximation,  the  asymptotic  result  should  only  be 
used  when  time  exceeds,  say,  20  correlation  time  intervals.  These  variance 
values  are  shown  in  Table  1. 

10.  Autocovariance  of  an  Integrated  Process 

The  variable  (yt)  characterizing  the  integral  of  a  stationary  stochastic 
process  is  given  in  discrete  form  by  equation  (22).  Suppose  that  successive 
values  of  y^.  are  formed  from  disjoint  sums  of  n  successive  values  of  the 
(xt)  series  thus 

yj  =  Ei=l  Xi+(n-l) j  '  (38) 
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TABLE  1 


i 


VARIANCES  FOR  THE  ESTIMATE  OF  THE  MEAN 
OF  A  FIRST-ORDER  STOCHASTIC  PROCESS 


Parameters: 

Correlation  time  (t*)  20  minutes 
Sampling  interval  (a)  1  minute 

Autocorrelation  constant  (b)  0.95 


Time,  tA 
(min) 

tA/x* 

V[xt]/ox2 

Uncor. 

V[Xt]/a2 

Asymp. 

v‘V\i 

2 

0.1 

0.9750 

0.5000 

20 

1.0 

0.7311 

0.0500 

— 

120 

6.0 

0.2723 

0.0083 

— 

240 

12.0 

0.1493 

0.0042 

0.1625 

480 

24.0 

0.0780 

0.0021 

0.0812 

960 

48.0 

0.0398 

0.0010 

0.0406 
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Note  that  y.  is  not  a  conventional  moving  average  of  the  x-series.  Since 

J 

{x^}  is  covariance  stationary  with  mean  zero,  {yt }  also  has  these  properties. 
Using  previous  notation,  the  autocovariance  of  the  {y^}  process  is  given  as 

Yyy^  ~  E[yj+k  yj]  '  (39^ 

For  notational  simplicity  the  number  (n)  of  members  of  (x^l  in  each  y  is 
not  noted  explicitly  here.  For  the  case  in  which  n=2,  from  (38), 

yl  =  X1  +  x2 

y2  =  x3  +  x4  (40) 

and  so  forth. 

Then, 

E[yj]=  E[x2  +  2XjX2  +  x2] 
or 


y  (0)  =  2y  (0)+2y  (l) 

yy  xx  ’xx 


If  {xtl  is  a  first-order  process,  from  (13), 


Yxx(1)  “  Yxx(0)b  • 


Then, 


Y  ( 0)/y  (0)  =  2  +  2b, 

yy  xx'  '  5 


which  agrees  with  (29)  for  the  special  case  of  t 
Pursuing  this  special  case  where  n  =  2, 

E [y1y23  =  e(xix3  +  X1X4  +  X2X3  +  X2X4] 
or 


=  2. 


YyyU)  -  Yxx(l)  +  2yxx(2)  +  Yxx(3)  . 
Similarly, 

Yyy(2)  =  yxx(3)  +  2xxx(4)  +  yxx(5) 
and  - 

V3)  =  YXX{5)  +  2^6)  +  ^(7)  • 


(41) 


(42) 


(43) 

(44) 

(45) 
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By  induction,  one  infers  that 

Yyy(j)  =  Yxx(2j-1)  +  Yxx(2j)  +  yxx(2j+1)  ,  j>l  ,  (46) 

for  this  case  (n=2) . 

11.  Consider  another  special  case  for  the  {y^.}  process  in  which  n=3  in 
equation  (38).  Following  the  same  procedure  used  in  developing  (46),  one 
finds 

Yyy(j)  -  Yxx(3j-2)  +  2yxx (3j-l) 

+  +  2Yxx(3j+1)  +  YxxOj+2)  .  (47) 

One  can  generalize  the  results  presented  this  far  by  applying  mathematical 
induction  to  the  case  in  which  n  is  not  assigned  a  specific  value  in 
equation  (38).  The  general  result  is 

vyy(j)  *  EU=1  k  Yxx(n(j-1)  +  k) 

+  Ek=l  k  vxx(r,(j+l)  -  k)  ,  j>l  .  (48) 

A  compact  expression,  which  is  equivalent  to  (48),  is 

Yyy(j)  ~  ^=0  Cpk  Yxx(n(j-l)+k+l)  , 

with 

cnk  =  |n  -  | int  [ ( n+1) / 2]  -  k||  ,  j>l  .  (49) 

Note  that  the  expressions  in  (48)  and  (49)  are  not  restrictive  regarding 
the  form  of  the  autocovariance  of  {x^.} .  Only  the  assumption  of  stationarity 
is  involved. 

12 .  A  Variance  Relationship  for  {y^} 

The  procedure  followed  in  deriving  the  relationship  between  the  auto¬ 
covariance  of  an  integrated  {y^.}  process  and  the  autocovariance  of  the  {x^.} , 
or  rate-,  process  involved  mathematical  induction  at  several  places.  A 
similar  procedure  is  followed  in  obtaining  the  textbook  formula  for  the 
variance  of  {yt> - 

vyy(0)  =  nyxx(0)  +  2Enk:j  (n-k)  Yxx(k)  .  (50) 
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13.  An  Autocorrelation  Relationship 

Often  the  analysis  of  an  autocorrelated  time  series  involves  a  comparison 
of  an  empirical  autocorrelation  function  with  several  theoretical  alternatives. 
To  facilitate  the  comparison,  autocorrelations  are  used  instead  of  autoco¬ 


variances.  The  following  autocorrelation  function  relating  the  {yt>  and  {x  } 
processes  is,  in  general,  rather  complicated.  Calculation  of  p  (j)  is  best 

done  via  computer.  The  computer  program  which  does  this  is  in^Annex  1. 

Since 


Cyy(j)  =  1yy(j)/ryy(0) 

Tyy(j)  Yxx(0) 

Yyy(0) 


Pyy(j) 


(51) 


But,  from  (50), 

V0)/Txx(0)  -  "  +  (n-k)PxJt(k)  . 
And,  from  (48), 

Yyy(j)/Yxx(0)  =  T ( n , j )  +  U(n, j )  , 

with 

T(n,j)  =  z*ml  k  Pxx(n(j-1)  +  k) 

U(n,j)  =  k  Pxx(n(j+1)  -  k)  . 

Finally,  from  (51,  52,  53), 

p  (j)  =  -T(n»j)  +  U(n,.i) 
yY  n  +  2.nk:J  (n-k)pxx(k) 

This  result  is  also  one  of  the  study  objectives. 


(52) 

(53a) 

(53b) 

(53c) 

(54) 
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14.  To  corroborate  the  expression  for  the  variance  of  an  integrated  first- 
order  process  --  equation  (31)  —  we  substitute 


Yxx(k)  -  bk  (55) 

into  (50),  obtaining 

Yyy(0)/cx2  .  n  +  .  (56) 

Apart  from  notational  differences,  equations  (31)  and  (56)  are  identical. 


15.  Autocorrelation  of  an  Integrated  First-Order  Process 

A  general  equation  for  the  autocorrelation  function  of  an  integrated 
first-order  process  is  obtained  by  substituting  b  for  p  (k)  in  equations 

AA 

(53)  and  (54).  This  operation  yields 

0  (,•)  =  C(n,b)  bn(J-‘>  H(n,H  bnJ 

pyyu;  n  +  2D(n,b) 


J>1 


(57a) 


where 


C(n,b)  =  i*  k  bk 


C(n,b) 


=  b(l-bn)  _  n  b 


n+1 


(l-b)‘ 


(1-b) 


(57b) 


and 


D(n,b)  =  (n-k)  bk 

n/_  u)  _  nb  b(l-bn) 
D(n,b)  1-b  -  (i_b)2  • 


(57c) 


As  an  example,  for  n=2. 


yy 


<j>  =  ^  b2j  ,  j>i 


(58) 


16.  The  Autospectrum 

The  discussion  of  autocovariance  and  autocorrelation  concerns  the 
temporal  relationship  between  members  of  a  time  series.  An  alternative 
means  of  studying  the  dynamics  of  a  stochastic  process  is  set  in  the 
frequency  domain.  The  variance  (or  power)  of  a  process  is  decomposed 
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into  the  variance  components  associated  with  each  of  its  Fourier  frequencies. 
This  form  of  expressing  process  dynamics  is  calculated  via  the  autospectrum 
of  the  process.  The  kind  of  question  answered  by  spectrum  analysis  is  the 
following:  How  does  the  variance  associated  with  process  fluctuations  of 
one  temporal  period  (or  reciprocal  frequency)  compare  with  those  of  other 
periods?  Altho  the  autospectrum  of  a  continuous  stochastic  process  x(t) 
can  be  defined  in  terms  of  the  Fourier  integral  of  x(t),  an  equivalent 
(  indirect")  method  is  used  here,  which  involves  the  autocovariance  of  x(t). 


17.  Notationally,  the  autospectrum  of  x(t)  is  expressed  as  a  function  of 
angular  frequency  to  as 


rix(“)  =  !  f  Txx(t)  cos(„t)dt  . 

J  0 

The  autospectrum  is  often  expressed  in  terms  of  the  natural  frequency 
v  in  cycles  per  unit  time: 

rxx('-> -  2”  r;x<“>  ■ 

From  p.  259,  Jenkins  and  Watts  (1968),  [3J,  the  smoothed  (or  estimated) 
autospectrum  of  the  discrete  process  {x^}  is 

rxx(k)e-j2”'’k4  , 

using  complex  notation  with  j  =  /Tf  . 

Alternatively, 

rxx^  =  2^xx(°)  +  2  zk=l  Yxx(k)cos  27rvkA]  »  0  1  v  1  1/ ( 2a)  . 


(59) 


(60) 


(61) 


The  weight  function  w(k),  of  lag  k,  is  bounded  between  zero  and  one.  When 
w(k)  is  set  to  unity  for  all  k,  the  unsmoothed  autospectrum  is  obtained. 

To  smooth  out  the  random  variation  occuring  in  the  estimate  of  y  (k),  a 
variety  of  forms  for  w(k)  may  be  used.  Some  are  suggested  in  [3]  and 


[3]  Jenkins,  G.M.  and  Watts,  D.G.  Spectral  Analysis  and  Its  Applications, 
c.  1968. 
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[4].  All  w(k)  have  the  property:  4 

w(k)  =0  ,  | k |  >  m  , 

for  some  integer  m. 

However,  we  are  only  concerned  with  the  theoretical  autospectrum  here,  since 
the  exact  expression  for  y  (k)  is  available.  Hence,  w(k)  will  be  set  to 
unity  in  analytic  applications.  When  calculating  a  spectral  density  via 
direct  numerical  evaluation  of  (61),  it  is  necessary  to  truncate  the 
infinite  sum  at  some  maximum  lag  m.  This  procedure  is  equivalent  to 
assigning  w(k)  the  boxcar  function: 
w(k)  =1  ,  | k |  £  m  , 

w(k)  =0  ,  |k|  >  m  . 

High  frequency  perturbations  are  superposed  on  the  spectrum  due  to  this 
truncation  —  the  Gibbs'  phenomenon  (p.  73,  [4]).  To  minimize  this  effect, 
the  boxcar  weight  function  (or  lag  window)  should  not  be  used.  A  better 
choice,  used  in  the  annexed  computer  program,  is  the  Tukey  lag  window. 

\  j 

18.  To  compare  the  autospectra  of  time  series  having  different  variance  i 

values,  it  is  convenient  to  use  a  normalized  autospectrum,  the  spectral 

density: 

Fxx(',)  -  rxx(',)/’'xx(0>  •  (62) 

From  (61)  and  (62)  , 

Fxx(v)  =  2a[1+2i“=1  w(k)pxx(k)cos  kA]  ,  0  <  kA  <  tt  (63) 

where 

A  =  2ttvA  .  (64) 


[4]  Hamming,  R.W.  Digital  Filters,  Prentice-Hall,  Englewood  Cliffs,  NJ, 
c.  1977. 


< 
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19.  Autospectrum  of  a  First-Order  Process 

Consider  (xt>  to  be  first-order  so  that  (13)  applies. 
Then,  with  (63), 


rxx(v)  =  2a[1+2z”_1  bkcos  kX]  .  (65) 

To  obtain  the  sum  in  (65),  one  can  use  the  following  result  from  Gradshteyn 
and  Ryzhik  (1965) ,  [5] : 


pkcos  kx  =  1-.PCPS  X  -  pncos 


„  I  1 

nx  +  p  cos  (n+1); 

x  +  p2 


1  -  2p  cos 


(66) 


For  |p |  <  1,  as  n  approaches  «>, 


z"=1  pkCOS  kx  =  —  P-(c°5  X  -  p) 

1  -  2p  cos  x  +  p 

Using  (67),  equation  (65)  becomes 
r„.(v)  -  '  b Z)  . 


(67) 


xx 


1  +  b  -  2b  cos  X 

Parenthetical ly,  the  last  result  could  have  been  obtained  by  another  method. 
This  method  uses  the  fact  that  a  first-order  time  series  is  a  first-order 
digital  filter  of  white  noise.  Thus, 


(68) 


Taking  Fourier  transforms  on  both  sides  of  this  equation  gives  the  transfer 
function  H (to ) ,  whose  square  is  given  by 

H^(w)  =  1/  [  1  +  b^  -  2  b  cos  to  A  ]  . 

One  then  uses  the  following  theorem  relating  the  spectra  of  the  {xt>  and 
(n^ }  processes  to  obtain  equation  (68): 

r;xU)  =  H^(w) r^n(u))  ,  0 _<  to  £  ir/A  , 

with 

rnn(w)  =  cr2A/ir  . 

Equation  (13b)  is  also  employed  to  obtain  the  spectral  density  in  (68). 


[5]  Gradshteyn,  I.S.  and  Ryzhik,  I.M.  Tables  of  Integrals,  Series,  and 
Products,  Academic  Press,  N.Y.,  c.  1965. 
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20.  Autospectrum  of  an  Integrated  First-Order  Process 

One  can  substitute  the  expression  for  p  ,  (57)  in  lieu  of  p  ,  into 

yy  xx 

the  general  equation  for  a  spectral  density  (equation  63).  This  process 
yields  an  equation  for  the  autospectrum  of  an  integrated  first-order 
process.  Except  for  special  values  of  n,  this  process  is  messy.  The 
computer  program  for  obtaining  numerical  results  is  found  in  Annex  1. 

The  particular  case  of  an  integrated  first-order  process  for  n=2  is 
tractable.  In  this  case,  p  is  given  by  (58).  Substitution  of  this 
expression  for  the  autocorrelation  into  (63),  with  w(k)  set  to  unity, 
yields 


Fy,W  =  24(1  +  ^ 
Using  (67)  with  (69), 


^k=l  ^  C0S  ^ 


r  (v)  =  24 [1  +  .(ltbIbicos_l,-,,b  ).  ,  . 
yy  l-2b^  cos  X  +  b4 


(69) 

(70) 


21.  Numerical  Results 

A  numerical  example  will  facilitate  the  comparison  of  properties  of  first- 
order  {xt>  with  integrated  first-order  {y^}  time  series.  Let  the 
parameters  b  =  0.9  and  a  =  1  minute.  Equations  (13a)  and  (58)  are  used  to 
calculate  autocorrelation  functions,  and  (68)  and  (70)  are  used  for  spectral 
densities.  Autocorrelations  are  tabulated  in  Table  2  and  plotted  in 
Figure  1.  The  autocorrelation  for  the  integrated  (n=2)  process  displays 
a  much  more  rapid  decrease  with  increasing  lag.  For  lags  k>_l,  the  auto¬ 
correlation  Pyy(k)  for  the  integrated  process  is  linear  on  a  log-uniform 
plot.  However,  unlike  pxx(k),  p  ( k )  has  a  change  of  slope  on  this  plot 
at  k=l.  The  autospectra  are  compared  in  Figure  2.  Notice  that  F^y(v)  is 
much  flatter  than  r  (v).  This  illustrates  the  approach  of  the  integrated 

XX 

process  toward  white  noise.  However,  for  this  degree  of  integration  (n=2) 
and  range  of  frequency  v,  the  two  autospectra  have  quite  similar  shapes. 

With  a  change  in  parameter  b  to  0.82  in  r  (v),  the  spectral  densities  are 

X  X 

nearly  conformable.  In  practical  terms,  this  implies  that  the  estimated 
autospectrum  of  an  integrated  (n=2)  first-order  process  would  be  nearly 
indistinguishable  from  that  of  a  first-order  process  for  certain  parameters. 
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TABLE  2 


COMPARISON  OF  AUTOCORRELATIONS  OF  A 

FIRST-ORDER  AND  AN  INTEGRATED* 

FIRST-ORDER  TIME  SERIES 

Autocorrelation  parameter  for  {x^.}:  b  =  0.9 

Lag,  k 

pxx(k) 

pyy(k) 

0 

1.0000 

1.0000 

1 

0.9000 

0.8550 

2 

0.8100 

0.6926 

3 

0.7290 

0.5610 

4 

0.6561 

0.4544 

5 

0.5905 

0.3680 

6 

0.5314 

0.2981 

10 

0.3487 

0.1283 

20 

0.1216 

0.0156 

*  Two  terms  are  in  the  sum. 


yj  "  Xj+1  +  xj+2  *  J'-°  • 

pxx(k)  .  bk 

PyyOO  =  ((l+b)/2)b2k_1 
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Figure  2.  Comparison  of  Spectral  Densities  for  a  First-Order  and  an  Integrated 
First-Order  Time  Series 
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22.  Summary 

This  report  has  examined  and  compared  statistical  properties  of 
stationary ,  first-order  time  series  and  integrated  first-order  series. 
A  connection  between  continuous  stochastic  processes  and  time  series 
was  made  thru  the  concept  of  correlation  time.  Expressions  for  the 
variance  of  the  sum  and  average  of  a  first-order  series  were  derived. 
Asymptotic  results  for  large  samples  were  shown  to  be  reasonable. 

When  {yt>  consists  of  disjoint  sums  of  any  stationary  process  {xt>, 
the  autocovariances  and  autospectra  of  these  processes  are  related. 
These  realtionships  were  derived,  and  were  particul arized  for  the  case 
in  which  (x^ }  is  first-order.  Numerical  examples  were  provided  and 
discussed.  A  computer  program  is  provided  to  evaluate  special  cases. 
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ANNEX  1 


COMPUTER  PROGRAMS 


This  annex  contains  the  computer  program  used  to  calculate  the 
autocorrel ation  function  and  spectral  density  of  a  general,  integrated 
first-order  stochastic  process.  The  main  program  (SA. DRIVER)  and 
subroutines  are  written  in  SIMSCRIPT  2.5.  They  have  been  run  on  the 
PRIME  550  minicomputer  in  an  interactive  mode.  However,  no  commands 
specific  to  PRIME  are  used.  The  program  listing  is  quite  English-like 
and,  with  internal  comments,  is  essentially  self-documenting.  Conversion 
to  Fortran  is  straightforward.  The  autocorrelation  functions  of  both 
the  x-series  --  p  (k)  --  and  integrated  x-series  (or  y-series)  are  sent 

XX 

to  the  terminal  for  output.  At  the  user's  option  the  spectral  densities 
of  the  x-  and  y-series  are  also  calculated  and  printed.  By  changing  only 
the  formula  for  p  (k),  one  can  obtain  these  statistical  properties  for 

A  A 

integrals  of  other- than-first-order  processes. 
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